Collaborative-filtered content recommendations with justification in real-time

ABSTRACT

Collaborative-filtered content recommendations with justification in real-time is described. A recommendation system determines these recommendations, in part, by identifying digital content items of a catalog that are associated with a single attribute used to describe digital content. The attribute used for the identification is based on affinity scores computed for a client device user to which the recommendations are being provided. These affinity scores indicate the client device user&#39;s affinity for different attributes used to describe the digital content. Once the digital content items are identified based on the one attribute, the recommendation system is then limited to ranking and selecting from the identified digital content items to provide the recommendations. The recommendation system does not process the entire catalog of digital content items at once to rank and select the items. Due to this, the described recommendation system performs less computing and is therefore faster than conventional recommendation systems.

BACKGROUND

Advances in computing systems enable content to be produced easily andalso to be distributed to end users through a variety of channels, e.g.,different web sites, different applications on a variety of devices, andso on. As a result, the amount of content available to users is not onlyis staggeringly large, but also continues to grow. Given the amount ofavailable content, users are unlikely to know about the entirety of thiscontent. To this extent, recommendation systems provide digital contentrecommendations to users for items these systems predict the users willlike. These recommendation systems provide digital contentrecommendations for a variety of items including, but not limited to,videos, music, audiobooks, e-books or periodicals, news articles,products or services, and so forth.

Broadly speaking, there are two types of conventional recommendationsystems. The first type of recommendation systems are item-based.Item-based recommendation systems generally provide a samerecommendation to different client device users that consume a samedigital content item. For instance, if two different users view a samevideo, item-based recommendation systems recommend a same set of videosto watch next. This approach does not personalize recommendations,however. Typically, personalized recommendations lead to higher userengagement and retention than non-personalized recommendations. Thesecond type of conventional systems, user-based recommendation systems,can provide personalized recommendations. In general, user-basedrecommendation systems learn a model for a given user's behavior, e.g.,based on the given user's interaction history with digital contentitems, purchase history, and so forth. In particular, these user-basedrecommendation systems learn this model in the form of latent factors.While such latent factors capture some aspect of user behavior, thecaptured aspect is often computer, but not human, interpretable. Due tocapturing aspects that are not human-interpretable, the recommendationsof user-based systems cannot be provided with correspondinghuman-interpretable justifications that explain why the recommendationsare provided. Though these recommendations recommend items that clientdevice users may find more interesting than other items, if therecommendations are heeded, client device users may not be comfortableinteracting with such recommendations because they lack correspondingjustifications. Accordingly, client device users may not interact withrecommendations provided using these conventional techniques.

SUMMARY

To overcome these problems, collaborative-filtered contentrecommendations with justification in real-time is leveraged in adigital medium environment. A recommendation system determines theserecommendations, in part, by identifying digital content items of acatalog that are associated with a single attribute that is usable todescribe digital content. The attribute used for the identification isbased on affinity scores that are computed for a client device user towhich the recommendations are being provided. These affinity scoresindicate the client device user's affinity for different attributes thatcan be used to describe the digital content. Once the digital contentitems of the catalog are identified based on the one attribute, therecommendation system is then limited to ranking and selecting from theidentified digital content items to provide the recommendations. Therecommendation system does not process the entire catalog of digitalcontent items at once to rank and select the items. Due to this, thedescribed recommendation system performs less computing thanconventional recommendation systems and is therefore able to providerecommendations faster than these systems.

This Summary introduces a selection of concepts in a simplified formthat are further described below in the Detailed Description. As such,this Summary is not intended to identify essential features of theclaimed subject matter, nor is it intended to be used as an aid indetermining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures.

FIG. 1 is an illustration of an environment in an example implementationthat is operable to employ techniques described herein.

FIG. 2 depicts an example implementation in which a digital contentdelivery system of FIG. 1 provides digital content recommendations withjustifications explaining why the recommendations are provided.

FIGS. 3A and 3B depict examples of user-to-content andcontent-to-attribute mappings, respectively.

FIG. 4 depicts an example user interface configured to present thedigital content recommendations with the justifications.

FIG. 5 depicts a procedure in an example implementation in which digitalcontent recommendations are provided with justifications to a clientdevice.

FIGS. 6A and 6B depict a procedure in an example implementation in whichscores indicative of affinities of client device users for contenthaving different attributes are computed and in which these affinityscores are used to collaboratively filter digital content of a catalogfor recommendation to the client device users.

FIG. 7 illustrates an example system including various components of anexample device that can be implemented as any type of computing deviceas described and/or utilized with reference to FIGS. 1-6 to implementembodiments of the techniques described herein.

DETAILED DESCRIPTION

Overview

As a result of advances in computing systems, the amount of contentavailable to users is not only is staggeringly large, but also continuesto grow. Given the amount of available content, users are unlikely toknow about the entirety of this content. To this extent, recommendationsystems provide digital content recommendations to users for items thesesystems predict the users will like. However, conventionalrecommendation systems suffer from a variety of drawbacks. One type ofthese systems, item-based recommendation systems, generally provide asame recommendation to different client device users that consume a samedigital content item. Accordingly, this approach does not personalizerecommendations, though personalized recommendations lead to higher userengagement and retention than non-personalized recommendations. Anothertype of these systems, user-based recommendation systems, providepersonalized recommendations, but do so using latent factors thatcapture some aspect of user behavior, which is often computer, but nothuman, interpretable. Consequently, recommendations from this secondtype of system cannot be provided with corresponding human-interpretablejustifications that explain why the recommendations are actuallyprovided. Due to these drawbacks, client device users may not interactwith recommendations provided using these conventional techniques.

To overcome these problems, collaborative-filtered contentrecommendations with justification in real-time is leveraged in adigital medium environment. In accordance with the described techniques,a recommendation system determines recommendations for a client deviceuser, in part, by identifying digital content items of a catalog thatare associated with a single attribute that can describe digitalcontent. The attribute used for the identification is based on affinityscores computed for the client device user, e.g., the attribute havingthe highest affinity score. Broadly speaking, the affinity scoresindicate the client device user's demonstrated affinity for differentattributes that can be used to describe digital content. By way ofexample, attributes of videos may correspond to genres, e.g., comedy,action, drama, romance, and so on. Accordingly, affinity scores may becomputed that indicate the client device user's affinity for comedyvideos, action videos, drama videos, romance videos, and so forth. Asdiscussed below, the attributes may describe a variety ofcharacteristics of recommendable items without departing from the spiritor scope of the described techniques.

In any case, the recommendation system may initially compute affinityscores for a client device user based on interactions of the user withdifferent content items. In one or more implementations, therecommendation system generates a matrix that plots client device usersagainst digital content items of a content catalog. The recommendationsystem populates cells of this matrix with strength-of-interest measuresfor the digital content items the client device users have interactedwith. These strength-of-interest measures represent a demonstratedinterest of the client device users with a particular content item. Ademonstrated interest may be explicit (e.g., user-provided ratings), ormay be estimated (e.g., based on an amount of interaction time with thecontent item). In addition to this user-item matrix, the recommendationsystem generates a matrix that plots the digital content items againstthe different attributes that are used to describe the digital content.The recommendation system populates this matrix with indications (e.g.,‘1’ or ‘0’) that indicate whether a digital content item is associatedwith a given attribute (e.g., ‘1’) or not (e.g., ‘0’). Based on thesematrices, the recommendation system computes the affinity scores, asdescribed in more detail below.

Given the affinity scores, the recommendation system is able todetermine attributes of the digital content items in which a clientdevice user demonstrates the most interest, e.g., the attributescorresponding to the highest affinity scores. Based on this knowledge,the recommendation system initially identifies the digital content itemsof the content catalog that are associated with one of the differentattributes, such as the attribute corresponding to a highest affinityscore of a client device user. Once the digital content items associatedwith this one attribute are identified, the recommendation system ranksthe identified digital content items—the recommendation system islimited to ranking solely the identified content items. In so doing, therecommendation system avoids ranking the entire catalog of digitalcontent items at once. The recommendation system then selects a subsetof the digital content items to recommend to the client device userbased on the ranking, e.g., the top-k digital content items. Therecommendation system can repeat this identifying, ranking, andselecting for additional attributes—one attribute at a time—according toa request specifying a number of recommendations associated with anumber of attributes.

In addition, the recommendation system generates justifications forrecommended items based on the attribute used to identify them. Considera scenario in which the recommendation system identifies digital contentitems that are associated with an ‘Action’ attribute, for instance,because affinity scores of a client device user indicate a demonstratedaffinity for items associated with the ‘Action’ attribute. This mayoccur when the highest affinity score computed for the client deviceuser corresponds to the ‘Action’ attribute. In this scenario, therecommendation system can generate a justification for the recommendeditems such as “Because you liked action items.” Obviously, this ismerely an example, and justifications can be generated to describe why auser is presented with recommendations in a variety of ways thatindicate the attribute used to identify the recommended items.Regardless of the particular attribute used for the identification, therecommendation system provides a justification with the selected subsetof content items based on the attribute used.

Accordingly, the described recommendation system supports severaladvantages. One advantage is that the recommendation system supportscollaborative filtering to provide recommendations that are diverse,personalized for a particular client device user, and serendipitous.Additionally, the techniques deployed by the recommendation systemenable the system to provide human-interpretable justificationsexplaining why these recommendations are provided. Due to this, thedescribed recommendation system may provide recommendations that clientdevice users are more comfortable interacting with (resulting inconversion) than recommendations provided by conventional techniques.Moreover, the recommendations provided by the described recommendationsystem may result in higher user engagement and retention than thoseprovided by conventional systems. Further, the described recommendationsystem performs less computing than conventional recommendation systemsbecause it ranks and selects items identified based on a singleattribute at a time. The described recommendation system is thus able toprovide recommendations after a request is received in substantiallyreal time, which is faster than conventional systems.

Term Descriptions

As used herein, the term “justification” refers to a human-interpretableexplanation describing why a recommendation is provided. In one or moreimplementations, justifications are configured as text stringsexplaining why particular recommendations are selected for provision toa client device user.

As used herein, the term “serendipitous” refers to a recommendation foran item a client device user has not demonstrated a particular affinitytoward but which the client device user does like, e.g., when the clientdevice user consumes the digital content.

As used herein, a “strength-of-interaction measure” refers to a measuredamount that a client device user demonstrates interest in or likes anitem. A strength-of-interest measure may be based on an explicitindication of the client device user's interest, such as a user-providedrating of the item. An example of this is selection of a number of starsout of a possible number of selectable stars. Alternately or inaddition, a strength-of-interest measure may implicitly measure theclient device user's interest based on behavior of the client deviceuser toward an item. Examples of this include, but are not limited to,an amount of time spent consuming (e.g., watching, listening to, and soon) a digital content item, a percentage of time spent consuming thedigital content item, a number of times the user consumes the digitalcontent item, and so forth.

As used herein, the term “attribute” of a digital content item refers toa characteristic of the digital content item that is describable inhuman language. By way of example, attributes for recommendable videoinclude genres, such as action, comedy, drama, romance, and so on.Accordingly, a video can be associated with a ‘Comedy’ attribute, e.g.,when the video is relatively funny. In the following discussion, thisvideo is said to be “associated with” the ‘Comedy’ attribute or “hasthe” ‘Comedy’ attribute. Indeed, a video may be associated with multiplegenre attributes, such as ‘Action’ and ‘Comedy’. Moreover, videos may beassociated with other attributes that describe characteristics of thevideos, such as describing subject matter of the videos, actorsappearing in the videos, types of videos, awards received, and so on.Indeed, the attributes used to describe videos may be different thanenumerated herein without departing from the spirit or scope of thedescribed techniques. Further still, various attributes may be used todescribe different types of recommendable items, e.g., audiobooks,e-books, products or services, music, and so forth, without departingfrom the spirit or scope of the techniques described herein.

As used herein, the term “content tag” refers to data that is usable todescribe an attribute of a recommendable item and/or data thatassociates an attribute with a recommendable item. An example of acontent tag is a text string, in metadata of a digital content item orassociated with the digital content item in some other way, that isindicative of the attribute. For instance, an action video may have inits metadata a text string ‘Action’; the presence of this text stringbeing effective to associate the video with the ‘Action’ attribute.Content tags may associate attributes with recommendable items indifferent ways without departing from the spirit or scope of thedescribed techniques.

In the following discussion, an example environment is first describedthat may employ the techniques described herein. Example implementationdetails and procedures are then described which may be performed in theexample environment as well as other environments. Consequently,performance of the example procedures is not limited to the exampleenvironment and the example environment is not limited to performance ofthe example procedures.

Example Environment

FIG. 1 is an illustration of an environment 100 in an exampleimplementation that is operable to employ collaborative-filtered contentrecommendations with justification in real-time as described herein. Theillustrated environment 100 includes a service provider system 102,client device 104, and digital content delivery system 106 that arecommunicatively coupled, one to another, via a network 108.

Computing devices that are usable to implement the service providersystem 102, client device 104, and digital content delivery system 106may be configured in a variety of ways. A computing device, forinstance, may be configured as a desktop computer, a laptop computer, amobile device (e.g., assuming a handheld configuration such as a tabletor mobile phone), and so forth. Thus, the computing device may rangefrom full resource devices with substantial memory and processorresources (e.g., personal computers, game consoles) to a low-resourcedevice with limited memory and/or processing resources (e.g., mobiledevices). Additionally, a computing device may be representative of aplurality of different devices, such as multiple servers utilized by abusiness to perform operations “over the cloud” as further described inrelation to FIG. 7.

The service provider system 102 is illustrated as including a servicemanager module 110 that is representative of functionality to provideservices accessible via the network 108 that are usable to make productsor services available to consumers. The service manager module 110, forinstance, may expose a website or other functionality that is accessiblevia the network 108 by a communication module 112 of the client device104. The communication module 112, for instance, may be configured as abrowser, a network-enabled application, and so on that obtains data fromthe service provider system 102 via the network 108. This data isemployed by the communication module 112 to enable a user of the clientdevice 104 to communicate with the service provider system 102 to obtaininformation corresponding to the products or services, e.g., web pageswith digital videos when the service provider system 102 is a videoplatform service.

In order to personalize the information provided to the client deviceusers, the service provider system 102 may employ a digital contentdelivery system 106. Although functionality of the digital contentdelivery system 106 is illustrated as separate from the service providersystem 102, this functionality may also be incorporated as part of theservice provider system 102, further divided among other entities, andso forth. The digital content delivery system 106 includes arecommendation manager module 114 that is implemented at least partiallyin hardware of a computing device to provide recommendations 116 ofdigital content with justifications 118 for those recommendations to theclient device 104. The justifications 118 indicate why therecommendations 116 are provided, e.g., “These video recommendations areprovided because User has demonstrated an interest in comedy videos.”The digital content delivery system 106 delivers the recommendations 116and the justifications 118 by configuring and transmitting communication120. In general, the communication 120 is configured as data that can beprocessed by the client device 104 to incorporate the recommendations116 and the justifications 118 into a user interface for output, e.g.,via a display of the client device 104, via speakers associated with theclient device 104, and so forth.

The recommendation manager module 114 is illustrated with affinitymodule 122 and recommendations generation module 124. The affinitymodule 122 and the recommendations generation module 124 representfunctionality to carry out aspects of recommendation—andjustification—provision based on digital content of digital contentcatalog 126 and user data 128, which are illustrated in storages 130,132, respectively. Here, the digital content catalog 126 is illustratedwith digital content item 134 having content tags 136. The digitalcontent catalog 126 is further depicted with ellipses to represent thatthe digital content delivery system 106 includes or has access tomultiple digital content items 134.

In accordance with the described techniques, the digital content items134 may assume a variety of forms, including, but not limited to digitalcontent including videos, music, audiobooks, digital books orperiodicals, news articles, web pages for products or services,advertisement components for products or services, and so forth. To thisend, the recommendations 116 are each configured to recommend to a userof the client device 104 a respective video, song, audiobook, digitalbook or periodical, news article, product or service, and so on. Thecontent tags 136 represent information describing the respective digitalcontent item 134, such as categories for describing digital content of atype or other pertinent information. In connection with videos, forinstance, the content tags 136 can be configured to describe categoriessuch as genre (e.g., comedy, action, and romance), subject matter (e.g.,floods, cats, and dogs), type (e.g., movie, television, and news video),and so forth. The content tags 136 can also be configured to describeother aspects of videos, such as actors, directors, producers, reviews,length, and so on. Indeed, the content tags 136 can be formatted indifferent ways and used to describe a variety of attributes of digitalcontent items 134 without departing from the spirit or scope of thetechniques described herein.

Data may also be generated based on the provision of the digital contentitems 134 to describe which users received these items as well asinteractions of the users with the items. This generated data may beincluded in user profile 138, for example. The user profile 138represents a single client device user, e.g., a client device userhaving a user account with the digital content delivery system 106, aclient device user authenticated to the client device 104, and/or aclient device user the digital content delivery system 106 otherwisetracks across various interactions with the client device 104. The userdata 128 is depicted with ellipses to indicate that the storage 132 isconfigured to store information about multiple users. As describedherein, the affinity module 122 and the recommendations generationmodule 124 use the interaction information 140 along with the contenttags 136 of the digital content items 134 to determine the digitalcontent items 134 to recommend and the justifications 118 forrecommending the recommended items.

In particular, these modules use a strength-of-interaction measure 142to carry out the functionality described above and below. Broadlyspeaking, the strength-of-interaction measure 142 measures an amountthat a client device user demonstrates interest in or likes a digitalcontent item 134. By way of example, the strength-of-interaction measure142 may be based on an explicit indication of a client device user'sinterest, such as a user-provided rating, e.g., a number of stars out ofa possible number of stars. Alternately or in addition, thestrength-of-interaction measure 142 may implicitly measure interestbased on behavior of a client device user, such as based on an amount oftime spent consuming (e.g., watching, listening to, and so on) a digitalcontent item, a percentage of time spent consuming the digital contentitem, a number of times the user consumes the digital content item, andso forth. In one or more implementations, the strength-of-interactionmeasure 142 may be a combination of multiple different aspectsindicative of a client device user's interest in a digital content item.Accordingly, the strength-of-interaction measure 142 may measure orestimate user interest in a digital content item in a variety of wayswithout departing from the spirit or scope of the techniques describedherein.

To generate the recommendations 116 and the justifications 118 for aparticular user, initially the affinity module 122 determines orotherwise obtains the strength-of-interaction measure 142 for each ofthe digital content items the particular user has consumed. Thesestrength-of-interaction measures 142 are determined for digital contentitems that are pertinent to the digital content being recommended. Ifthe digital content being recommended is videos, for instance, then thestrength-of-interaction measures 142 are determined for videos theparticular user has previously watched. Similarly, if the digitalcontent being recommended is audiobooks, then thestrength-of-interaction measures 142 are determined for audiobooks towhich the particular user has previously listened.

In one or more implementations, the affinity module 122 generates amatrix based on these strength-of-interaction measures 142, such thatthe rows of the matrix correspond to client device users and the columnscorrespond to digital content items. Alternately, the columns of thematrix correspond to the client device users and the rows correspond tothe digital content items. Here, the cells of the matrix are thestrength-of-interaction measures 142. Accordingly, when the rowscorrespond to client device users and the columns correspond to digitalcontent items, a row indicative of an individual user will include thestrength-of-interaction measures 142 determined for the individual userin relation to each of the digital content items, and another rowindicative of a different individual user will include thestrength-of-interaction measures 142 determined for the differentindividual user in relation to each of the digital content items. Itfollows then that a column indicative of an individual digital contentitem includes the strength-of-interaction measures 142 for each of theclient device users that consumed the item.

The affinity module 122 also generates a mapping of selected contentattributes to the digital content items 134 of the digital contentcatalog 126 based on the content tags 136. Consider an example in whichthe affinity module 122 generates the mapping to map a catalog ofdigital movies to categories indicated by the content tags 136, such asgenre and actors appearing in the videos. In this example, the videosmay be associated with content tags such as ‘action,’ drama,“romance,”as indicative of genre and tags such as ‘Sarah Jessica Parker,’ ‘HughJackman,’ as indicative of actors appearing in the videos. In a similarfashion as the strength-of-interaction measures 142 are mapped to usersusing a matrix, in one or more implementations, the selected attributesare also mapped to the digital content items 134 using a matrix.

Referring again to the video example, the affinity module 122 generatesa second matrix based on the digital content items 134 and therespective content tags 136. The affinity module 122 can configure thissecond matrix such that the rows of the matrix correspond to the digitalcontent items 134 of the digital content catalog 126 and the columnscorrespond to the selected attributes. Alternately, the affinity module122 can configure this second matrix such that the columns of the matrixcorrespond to the selected attributes and the rows correspond to thedigital content items 134. Here, the cells of the matrix may simply beindications regarding whether the digital content items have a tag thatindicates association with each of the attributes or not, e.g., thevalue of a cell may be ‘1’ to indicate that a digital content item 134has a content tag 136 indicative of the attribute (the presence of a tagdescribing a video as an ‘action’ video results in a ‘1’ for anaction-attribute column) and a ‘0’ to indicate that the digital contentitem does not have a content tag 136 indicative of the attribute (theabsence of a tag describing a video as an ‘action’ video results in a‘0’ for the action-attribute column). The association or non-associationwith an attribute can be indicated in other ways than one and zero,respectively, without departing from the spirit or scope of thetechniques described herein.

In any case, the affinity module 122 is configured to determineaffinities of a user in real-time for the different selected attributes,e.g., by computing a product of the above-described user-content matrixand the content-attribute matrix. Based on these determined affinities,the recommendations generation module 124 determines the digital contentitems 134 to provide to the client device 104 as the recommendations116. To determine the recommendations 116, the recommendationsgeneration module 124 filters the digital content items 134 of thedigital content catalog 126 based on the determined affinities for theclient device user. In one or more implementations, the recommendationsgeneration module 124 does this by initially identifying from thedigital content catalog 126 the digital content items 134 having aselected attribute for which the user is determined to have an affinity,e.g., a selected attribute for which the user has a highest determinedaffinity, the top-k selected attributes according to the determinedaffinities, the selected attributes for which the determined affinitymeets a predetermined threshold, and so forth. Consider an example inwhich the affinity module 122 determines that a user of the clientdevice 104 has a highest affinity for digital content items 134 with thecontent tag 136 ‘Finance’. In this example, the recommendationsgeneration module 124 initially identifies each of the digital contentitems 134 having a ‘Finance’ content tag 136 in real-time. Once theseare identified, the recommendations generation module 124 ranks theidentified digital content items 134 in real-time based on thedetermined affinities of the client device user and characteristics ofthe identified items, as indicated by the content tags 136.

Unlike conventional techniques, the described techniques enable thedigital content delivery system 106 to generate the recommendations 116using collaborative filtering and also provide the recommendations 116with the justifications 118. In contrast to this, conventionalrecommendation techniques leveraging collaborative filtering are basedon latent factors that are capable of providing recommendations matchinga user's demonstrated tastes but are not human-interpretable.Consequently, conventional recommendation techniques that usecollaborative filtering do not also provide justifications—thatcorrespond to the leveraged latent factors—for the recommendations.Because client device users can be leery of interacting with recommendeddigital content that is not presented with a justification, manyconventionally configured systems do not utilize collaborativefiltering. These conventional systems can thus fail to providerecommendations for digital content that is diverse, personalized for aparticular client device user, and serendipitous. Using the describedtechniques, however, the recommendation manager module 114 is able togenerate justifications as human-interpretable language (e.g., text) forrecommendations determined with collaborative filtering, e.g., includingrecommendations for digital content that is diverse, personalized for aparticular client device user, and serendipitous. As used herein,“serendipitous” refers to a recommendation for an item a client deviceuser has not demonstrated a particular affinity toward but which theclient device user does like, e.g., when the client device user consumesthe digital content. Accordingly, the recommendation manager module 114may cause more client device users to interact with the recommendationsprovided than conventional recommendation techniques.

Having considered an example environment, consider now a discussion ofsome example details of the techniques for rule determination ofcollaborative-filtered content recommendations with justifications inreal-time in a digital medium environment in accordance with one or moreimplementations.

Collaborative-Filtered Content Recommendations with Justifications

FIG. 2 depicts an example implementation 200 in which a digital contentdelivery system of FIG. 1 provides digital content as recommendationswith justifications explaining why the recommendations are provided. Theillustrated example 200 includes from FIG. 1 the affinity module 122 andthe recommendations generation module 124.

The affinity module 122 is depicted obtaining thestrength-of-interaction measures 142 and content tag data 202. Thecontent tag data 202 is indicative of the content tags 136 with whichthe digital content items 134 of the digital content catalog 126 aretagged. In the illustrated example, the affinity module 122 is depictedwith matrix generation module 204, which represents functionality to mapusers to the digital content items 134 of the catalog and to mapselected attributes indicated by the content tags 136 to the digitalcontent items 134. In one or more implementations, the matrix generationmodule 204 memorializes these mappings by generating user-item matrix206 and item-attribute matrix 208. In this context, consider FIGS. 3Aand 3B.

FIGS. 3A and 3B depict at 300 examples of user-to-content andcontent-to-attribute mappings. In particular, FIG. 3A depicts auser-to-content mapping configured as a first matrix 302 and FIG. 3Bdepicts a content-to-attribute mapping configured as a second matrix304.

In the illustrated example 300, the rows of the first matrix 302correspond to users 306 and the columns correspond to content items 308.In accordance with the described techniques, the users 306 are eachassociated with a user profile 138 and the content items 308 representthe digital content items 134 of the digital content catalog 126.Accordingly, the interaction information 140 of these users 306describes interactions with the content items 308. In particular, theinteraction information 140 includes a measure of the interactionstrength, e.g., strength-of-interaction measures 142, that indicate arelative strength of a client device user's interaction with thedifferent content items 308. As noted above, the strength-of-interactionmeasure 142 may be configured in a variety of ways (e.g., express ratinggiven by a user, pro-rata amount of consumption, etc.) without departingfrom spirit or scope of the techniques described herein.

Regardless of the particular measure, the cells of the first matrix 302represent strength-of-interaction measures 142 indicated by a respectiveuser's interaction information. For instance, a value of first cell 310(‘0.982’) corresponds to the strength-of-interaction measure 142included in the user profile 138 of the user ‘kaa$h17’ for the contentitem ‘Content_1’. Similarly, a value of second cell 312 (‘0.743’)corresponds to another strength-of-interaction measure 142 included inthe user profile 138 of the user ‘kaa$h17’ for the content item‘Content_2’. In accordance with the described techniques, the matrixgeneration module 204 processes the strength-of-interaction measures 142to generate the user-item matrix 206 so that it is configured in asimilar manner as the first matrix 302, e.g., one axis corresponding toclient device users, the other corresponding to items of a catalog, andcells corresponding to respective strength-of-interaction measures ofthese client device users with these items.

In contrast, the rows of the second matrix 304 correspond to the contentitems 308 and the columns correspond to selected attributes 314 ofcontent items. In accordance with the described techniques, the selectedattributes 314 correspond to attributes of content items that aredescribable using the content tags 136. In particular, the content tags136 represent the attributes with which a content item is associated,e.g., if one attribute that can be used to describe content is ‘Action’,the presence of an ‘Action’ content tag 136 indicates that therespective content item is an action content item or has the attributeof being associated with action. To this end, the cells of the secondmatrix 304 include values indicative of whether a corresponding contentitem 308 is associated with the selected attributes 314 or not, asindicated by the content tags 136.

In this example 300, a value of one indicates that a content item isassociated with the attribute and a value of zero indicates that thecontent item is not associated with the attribute. For instance, thirdcell 316 has a value of one. This indicates that the content item 308‘Content_1’ is tagged with an ‘Action’ content tag 136, which indicatesthat ‘Content_1’ is associated with the action attribute. In contrast,fourth cell 318 has a value of zero. This indicates that the contentitem 308 ‘Content_1’ is not tagged with a ‘Drama’ content tag 136,indicating therefore that ‘Content_1’ is not associated with the dramaattribute. In accordance with the described techniques, the matrixgeneration module 204 processes the content tag data 202 to generate theitem-attribute matrix 208 so that it is configured in a similar manneras the second matrix 304, e.g., one axis corresponding to the items ofthe digital content catalog 126, the other corresponding to selectedattributes that can be used to describe the items, and cellscorresponding to values indicative of association or non-association ofthe items with the selected attributes as indicated by the content tags136.

The affinity module 122 is further depicted having user-attributescoring module 210 in the illustrated example 200. The user-attributescoring module 210 represents functionality to compute attributeaffinity scores 212 for client device users based on the user-itemmatrix 206 and the item-attribute matrix 208. In general, the attributeaffinity scores 212 represent a given client device user's affinity fordigital content items associated with a particular attribute. Withreference to the attributes illustrated in FIG. 3B, for instance, theattribute affinity scores 212 are configured to indicate a particularuser's affinity for content associated with the ‘Action’ attribute. Togenerate these attribute affinity scores 212, the user-attribute scoringmodule 210 computes a product of the user-item matrix 206 and theitem-attribute matrix 208. In one or more implementations, theuser-attribute scoring module 210 further computes Log-Likelihood Ratiosfrom the multiplied matrices to derive the attribute affinity scores212. The user-attribute scoring module 210 also represents functionalityto configure the attribute affinity scores 212 as latent factorvectors—generating one such vector for each client device user. Withrespect to FIGS. 3A and 3B, attribute affinity scores configured aslatent factor vectors indicate for each of the users 306 their affinityscores for ‘Action’, ‘Drama’, ‘Romance’, ‘Sarah Jessica Parker,’ and‘Attribute_n’.

The recommendations generation module 124 is depicted obtaining theattribute affinity scores 212, e.g., obtaining the per-user latentfactor vectors. From the attribute affinity scores 212, therecommendations generation module 124 is configured to determine therecommendations 116. To determine the recommendations 116, therecommendations generation module 124 employs collaborative filteringmodule 214. Initially, the collaborative filtering module 214 identifiesthe digital content items 134 of the digital content catalog 126 havinga particular attribute, e.g., an attribute corresponding to a highestattribute affinity score 212 for a client device user underconsideration. Consider an example in which a highest of the attributeaffinity scores 212 for a given client device user corresponds to the‘Action’ attribute. In this scenario, the collaborative filtering module214 initially identifies each digital content item of the digitalcontent catalog 126 having an ‘Action’ content tag 136. Thecollaborative filtering module 214 then ranks the identified digitalcontent items.

In one or more implementations, the collaborative filtering module 214ranks the identified digital content items based on the attributeaffinity scores 212 for the client device user under consideration andthe attributes with which each of these digital content items isassociated, e.g., according to the content tags 136. These attributeaffinity scores may be represented, for instance, using the latentfactor vector generated for the client device user under consideration.The collaborative filtering module 214 may further utilize latent factorvectors that are generated for the digital content items from theuser-item matrix 206 using a matrix factorization algorithm, such asAlternating Least Squares. In such scenarios, the collaborativefiltering module 214 computes an identified content score as a dotproduct of the client device user's latent factor vector and the latentfactor vector of an identified digital content item. The collaborativefiltering module 214 then ranks the identified digital content items forthe user under consideration according to these dot products. For therecommendations 116, the collaborative filtering module 214 selects thetop-k ranking digital content items.

These top-k digital content items correspond to items identified asassociated with one particular attribute. This particular attributefurther serves as a basis for generating the justifications 118 of thesetop-k items, such that if the attribute relative to which these itemsare initially identified is ‘Action’ then these items can be presentedwith a justification such as ‘Because you demonstrated an interest inaction items’. The number of items selected in relation to a singleattribute may be based on a user interface via which the items are beingpresented, e.g., a form factor of the client device 104, display area ofan application's user interface, and so forth.

The collaborative filtering module 214 is capable of performing thisidentifying, ranking, and selecting of content items for each of thedifferent selected content attributes. However, the collaborativefiltering module 214 may limit the identifying, ranking, and selectingto a subset of the attributes, such as a highest scoring attribute(according to the attribute affinity scores), a top-k number ofattributes, and so forth. Like the number of items selected forrecommendation with one particular attribute, a number of attributes forwhich items are identified may be based on a user interface, e.g., aform factor of the client device 104, display area of an application'suser interface, and so forth. As items are identified, ranked, andselected for recommendation relative to a particular attribute, thecollaborative filtering module 214 uses that particular attribute toserve as a basis for generating the corresponding justifications 118.The recommendations generation module 124 generates the communication120 with the recommendations 116 and the justifications 118 determinedin this way.

In one or more implementations, the recommendations generation module124 generates the communication 120 and communicates it to the clientdevice 104 as content items are recommended in relation to eachattribute. Alternately or in addition, the recommendations generationmodule 124 generates the communication 120 and communicates it to theclient device 104 for content items recommended in relation to multipleattributes. In any case, the communication 120 enables the client device104 to present the recommendations 116 with the justifications 118. Inthis context, consider FIG. 4.

FIG. 4 depicts an example user interface 400 that is configured topresent digital content recommendations with justifications indicatingwhy the recommendations are provided. The illustrated example 400includes recommendation presentation interface 402, which is illustratedas a graphical user interface though other configurations (e.g., audio)are contemplated in the spirit or scope of the techniques describedherein.

In accordance with the described techniques, the recommendationpresentation interface 402 is generated to present the recommendations116 along with the justifications 118. For example, the recommendationpresentation interface 402 includes video recommendations 404 and videojustification 406. The video justification 406 indicates to a user whythe video recommendations 404 are being presented via the recommendationpresentation interface 402. In this case, the video justification 406indicates that the video recommendations 404 are presented via the userinterface ‘Because You Liked Comedy’. This represents that the attributeaffinity scores 212 computed for indicated user 408 (‘kaa$h17’) mayinclude a highest affinity score for ‘Comedy’ videos.

The other recommendations presented via the recommendation presentationinterface 402 also include justifications. The second row of videorecommendations, for instance, is presented with the justification‘Because You Liked Drama’. In contrast to the video justification 406and the justification for the second row of videos, the third row ofvideo recommendations represents a predicted genre of videos (e.g.,romantic comedy “Rom-Com”) with the justification ‘Because You LikedComedy & Drama’. This indicates that the recommendations generationmodule 124 is configured to predict content for which the indicated user408 will have an affinity based on a combination of multipleattributes—in this case the attributes of being associated with comedyand being associated with drama.

Broadly speaking the recommendations generation module 124 is configuredto predict content for which a user will have an affinity but for whichthe user has not yet demonstrated an affinity. Invariably, most userswill not have consumed each digital content item 134 of the digitalcontent catalog 126. As part of predicting attributes (and thus content)for which a user will have an affinity, the affinity module 122 augmentsthe user-item matrix 206 to include strength-of-interaction measures forthe digital content items 134 a user has not yet consumed. In one ormore implementations, the affinity module 122 determines at least oneother user that is similar to a user for which the user-item matrix 206is being completed. The affinity module 122 copiesstrength-of-interaction measures from the at least one other user'smatrix to the user-item matrix 206 being completed. The affinity module122 may augment the user-item matrix 206 to complete it in other waywithout departing from the spirit or scope of the techniques describedherein.

Given this, the user-attribute scoring module 210 can re-compute theattribute affinity scores 212 with the augmented user-item matrix 206,e.g., by multiplying the augmented user-item matrix 206 and theitem-attribute matric 208 and computing a Log-Likelihood Ratio based onthese multiplied matrices. The collaborative filtering module 214 canthen perform the above described identifying, ranking, and selecting topredict content a user will like even though the content has attributesfor which the user has not yet demonstrated an affinity. This enablesthe recommendations generation module 124 to provide the recommendations116 with the justifications 118 to client device users that haveconsumed a relatively small amount of content, e.g., new users to aplatform. This also enables the recommendations generation module 124 torecommend newly released (e.g., posted, published, and so on) digitalcontent items, digital content items having an attribute the user ispredicted to have an affinity for, and so forth. This is because therecommendation manager module 114 exploits the metadata (e.g., contenttags 136) of recommendable items rather than recommending items based onnumbers of client device users having consumed the items. Recommendingitems based on metadata and determined user affinities can be especiallybeneficial for time-sensitive content such as news content, where itemsmay be considered stale by the time enough client device users haveconsumed an item to cause it to be “popular.”

As noted above, the collaborative filtering module 214 filters contentattribute-by-attribute for recommendations, e.g., identifying digitalcontent items of the digital content catalog 126 that are associatedwith one attribute and then next attributes (one attribute at a time) ifmore recommendations are requested. In so doing, the recommendationsgeneration module 124 avoids computing a dot product or a customFoundation Matrix score (FM-score) over an entirety of the digitalcontent catalog 126. Limiting dot product computations to the digitalcontent items associated with one attribute enables the recommendationsgeneration module 124 to select items for recommendation moreefficiently (reduced computations) and faster than conventionalrecommendation techniques. This enables the recommendation managermodule 114 to provide the recommendations 116 in substantially real-timeresponsive to a request for the recommendations 116, e.g., a requestfrom the client device 104 made via an application programming interface(API). Moreover, the reduction in the computations enables therecommendation manager module 114 to scale its recommendation provisionto a larger number of client device users than conventional techniqueswhile maintaining substantially real-time delivery.

Having discussed example details of the techniques forcollaborative-filtered content recommendations with justification inreal-time, consider now some example procedures to illustrate additionalaspects of the techniques.

Example Procedures

This section describes example procedures for collaborative-filteredcontent recommendations with justification in real-time in one or moreimplementations. Aspects of the procedures may be implemented inhardware, firmware, or software, or a combination thereof. Theprocedures are shown as a set of blocks that specify operationsperformed by one or more devices and are not necessarily limited to theorders shown for performing the operations by the respective blocks. Inat least some implementations the procedures are performed by a suitablyconfigured device, such as the digital content delivery system 106 ofFIG. 1 that makes use of an affinity module 122 and recommendationsgeneration module 124.

FIG. 5 depicts an example procedure 500 in which digital contentrecommendations are provided with justifications to a client device.Affinity scores are obtained that are indicative of a client deviceuser's affinities for different attributes of recommendable digitalcontent (block 502). By way of example, the recommendations generationmodule 124 obtains the attribute affinity scores 212. As discussedabove, these attribute affinity scores 212 are indicative of a clientdevice user's affinities for different attributes of recommendabledigital content, e.g., the digital content items 134 of the digitalcontent catalog 126.

A first set of digital content items of a content catalog are identifiedthat are associated with a first of the different attributes (block504). In accordance with the principles discussed herein, the digitalcontent items are identified according to metadata describing theseitems. Further, the one attribute used as a basis for identifying theseitems is based on the obtained affinity scores, e.g., an attributecorresponding to the highest affinity score. By way of example, thecollaborative filtering module 214 identifies one or more of the digitalcontent items 134 of the digital content catalog 126 that are associatedwith one of the different attributes, such as an attribute having ahighest of the attribute affinity scores obtained at block 502. Thecollaborative filtering module 214 identifies these digital contentitems according to the content tags 136—the identified items having acontent tag that associates the identified items with the one differentattribute.

The first set of digital content items are ranked based, in part, on theobtained affinity scores and the metadata of the first set of digitalcontent items (block 506). By way of example, the collaborativefiltering module 214 ranks the digital content items identified at block504. The collaborative filtering module 214 ranks these identified itemsbased, in part, on the affinity scores obtained at block 502 and thecontent tags 136. As discussed above and below, the affinity scores areindicated by a latent factor vector computed for the client device user,and the content tags of the identified items are indicated by latentfactor vectors computed for the identified content items.

One or more of the first set of digital content items are selected forrecommending to the client device user based on the ranking (block 508).By way of example, the collaborative filtering module 214 selects one ormore of the digital content items identified at block 504 based on theranking at block 506, such as the top-k ranked items. Therecommendations generation module 124 configures information about theseitems for provision as the recommendations 116. In other words, therecommendations 116 are formed to represent the items selected at block508.

A justification is generated based on the first attribute and explainswhy the recommendations are provided (block 510). In accordance with theprinciples discussed herein, the justification is provided with therecommendations. By way of example, the recommendations generationmodule 124 generates the justifications 118 for the items selected atblock 508. The justifications 118 explain why those items arerecommended to the client device user via the recommendations 116. Therecommendations generation module 124 further provides thejustifications 118 with the recommendations 116, e.g., as part of thecommunication 120.

Determine whether more recommendations are requested (block 512). If adetermination is made that more recommendations are requested (i.e.,“YES” at block 512), the method returns to block 504 to identify atleast a second set of digital content items that are associated with atleast a second of the different attributes, e.g., corresponding to atleast a second-highest affinity score. By way of example, therecommendations generation module 124 determines that morerecommendations are requested than items selected at block 508 and themethod returns to block 504 to identify digital content items associatedwith another attribute, e.g., a next-highest scoring attribute. If adetermination is made that more recommendations are not requested (i.e.,“NO” at block 512), however, then the recommendations are presented withthe justifications (block 514). By way of example, the recommendationsgeneration module 124 determines that more recommendations are notrequested, e.g., the number of items selected equals or exceeds a numberof items requested. The digital content delivery system 106 delivers therecommendations 116 with the justifications 118 to the client device104, which includes functionality operable to present therecommendations 116 with the justifications 118. For instance, theclient device 104 presents the recommendation presentation interface 402having the video recommendations 404 and the video justification 406. Asdiscussed above, the recommendations 116 may recommend items other thanvideos, such as music, e-books, audiobooks, products and services forpurchase, and so on.

FIGS. 6A and 6B depict an example procedure 600 in which scoresindicative of affinities of client device users for content havingdifferent attributes are computed and in which these affinity scores areused to collaboratively filter digital content of a catalog forproviding recommendations to the client device users in real-time.

A user-item matrix is generated having cells that indicatestrength-of-interaction measures of client device users with digitalcontent items of a catalog of the digital content items (block 602). Byway of example, the affinity module 122 generates the user-item matrix206 based on the strength-of-interaction measures 142 taken from theuser data 128. As noted above, the strength-of-interaction measure 142measures one or more interactions of a user with a digital content item134 of the digital content catalog 126—some measures may correspond toan explicitly expressed indication (e.g., a rating) of interest in acontent item while other measures estimate the interest in the content(e.g., a length of time of the interaction).

An item-attribute matrix is generated having cells that indicateassociation of the digital content items of the catalog with differentattributes that can be used to describe the digital content items (block604). By way of example, the affinity module 122 generates theitem-attribute matrix 208 for the digital content items 134 of thedigital content catalog 126. Based on the content tags 136, the affinitymodule 122 sets cell values of the item-attribute matrix 208 to indicatewhether a digital content item is associated with the differentattributes or not.

The user-item matrix and the item-attribute matrix are multiplied (block606). By way of example, the user-attribute scoring module 210multiplies the user-item matrix 206 and the item-attribute matrix 208.Attribute affinity scores are determined for the client device users bycomputing a Log-Likelihood Ratio (LLR) with the multiplied matrices(block 608). By way of example, the user-attribute scoring module 210determines the attribute affinity scores 212 by computing an LLR withthe user-item matrix 206 and the item-attribute matrix 208 multiplied atblock 606.

Digital content items associated with an attribute are identified for aparticular client device user (block 610). In accordance with theprinciples discussed herein, the attribute has a highest attributeaffinity score for the particular client device user. By way of example,the collaborative filtering module 214 identifies the digital contentitems 134 of the digital content catalog 126 that are associated with ahighest-scoring attribute determined for a user of the client device104. The procedure 600 continues at ‘A’ from FIG. 6A to FIG. 6B.

A latent factor vector is determined for the particular client deviceuser and latent factor vectors are determined for the identified digitalcontent items (block 612). In accordance with the principles discussedherein, the latent factor vector for the particular client device useris determined based on the attribute affinity scores and the latentfactor vectors for the identified digital content items are determinedbased on strength-of-interaction measures as indicated by the user-itemmatrix 206. By way of example, the recommendations generation module 124determines a latent factor vector for a user of the client device 104.The recommendations generation module 124 determines the latent factorvector of the client device 104's user based on the attribute affinityscores 212 computed for the user. The recommendations generation module124 also determines latent factor vectors for the digital content itemsidentified at block 610. In one or more implementations, therecommendations generation module 124 determines the latent factorvectors for these identified digital content items by using a matrixfactorization algorithm (e.g., Alternating Least Squares) in relation tothe user-item matrix 206.

The identified items are ranked based on dot products of the latentfactor vector of the particular client device user and the latent factorvectors of the identified digital content items (block 614). By way ofexample, the collaborative filtering module 214 computes dot products ofthe client device user's latent factor vector and the latent factorvectors of the identified digital content items. The collaborativefiltering module 214 then ranks the identified items based on thecomputed dot products.

A top-k ranked items are selected to provide as recommendations to theparticular client device user (block 616). By way of example, thecollaborative filtering module 214 selects a top-k ranked items toprovide as the recommendations 116 to a user of the client device 104.In one or more implementations, k corresponds to a number equal to orgreater than one that is controlled by a user interface of the clientdevice 104 via which the recommendations 116 are to be presented, e.g.,an application programming interface (API) associated with the userinterface may request k recommendations in relation to an attribute.

The user-item matrix is completed for at least one client device user(block 618). In accordance with the principles discussed herein, thiscompletion includes augmenting the user-item matrix's cells thatcorrespond to digital content items the at least one client device userhas not consumed. In particular, the cells corresponding to thenot-yet-consumed digital content items are augmented to includepredicted strength-of interaction measures. By way of example, theaffinity module 122 completes the user-item matrix 206 for a user of theclient device 104. The affinity module 122 completes the user-itemmatrix 206 by augmenting the cells that correspond to the digitalcontent items the client device 104's user has not consumed. In one ormore implementations, the affinity module 122 augments the cells withstrength-of-interaction measures taken from other users that theaffinity module 122 determines similar to the user of the client device104.

Absent completion of the user-item matrix 206, the affinity module 122may not be able to identify affinities for attributes that collaborativefiltering techniques and low-rank matrix factorization can be leveragedto identify. By completing the user-item matrix 206, however, theaffinity module 122 is able to leverage such collaborative filteringtechniques and low-rank matrix factorization to identify affinities forsome attributes—the affinities identified using these techniquescontrast with affinities already demonstrated by the user according tothe digital content items the user has actually consumed. In otherwords, by completing the user-item matrix 206, the affinity module 122is not limited providing recommendations based solely on the limitedcontent items the user has consumed.

Blocks 606 through 616 are repeated using the completed user-item matrixin place of the user-item matrix (block 620). By way of example, therecommendation manager module 114 leverages the functionality of theaffinity module 122 and the recommendations generation module 124 toprovide predictive recommendations recommendations for content that isassociated with attributes the user of the client device 104 has not yetdemonstrated an affinity—by repeating block 606 through 616 using thecompleted user-item matrix in lieu of the user-item matrix 206. In oneor more implementations, the user-item matrix 206, before beingcompleted, includes strength-of-interaction measures 142 for the digitalcontent items 134 with which client device users have interacted. Forthe digital content items 134 the client device users have notinteracted with, the user-item matrix 206, before being completed,includes values or other indicator representative of thenon-interaction.

Consider an example recommendation scenario in which the recommendationmanager module 114 generates recommendations in connection with a numberof users, a number of digital content items, and a defined number ofattributes. In the following discussion, the number of users isrepresented by the term ‘M’, the number of digital content items isrepresented by the term ‘N’, and the number of attributes is representedby the term ‘A’. Further, the affinity module 122 may leverage a numberof latent factors, k, (e.g., 10-40 latent factors). In this example andthe following discussion, assume that the number of users M and thenumber of digital content items N, are orders of magnitude greater thanA or k.

In this example, the recommendation manager module 114 leverages thefunctionality represented by the affinity module 122 and therecommendations generation module 124 to implement a sparse matrixfactorization algorithm, e.g., Alternating Least Squares (ALS). Forinstance, the recommendation manager module 114 leverages thisfunctionality to implement the sparse matrix factorization algorithm inrelation to a rating matrix R having dimensions M×N. As part of this,the affinity module 122 initially generates two matrices—a firstelemental matrix U, having dimensions M×k, and a second elemental matrixV, having dimensions k×N. Given that the recommendation manager module114 generates the ratings matrix R as a sparse matrix, the affinitymodule 122 is employed to compute user-attribute affinities with LLR bymultiplying the ratings matrix R by the item-attribute matrix 208. Theruntime complexity for the affinity module 122 to perform suchcomputations is proportional to a number of ratings in the ratingsmatrix R and is not proportional to M×N, which is a much larger numberthan the number of ratings or non-zero entries in the ratings matrix R.

In the continuing scenario assume, however, that the affinity module 122completes this ratings matrix R to form completed ratings matrix R1,e.g., by multiplying the first elemental matrix U and the secondelemental matrix V. In this scenario, the cells (or almost all of thecells) of the completed ratings matrix R1 are populated with non-zeroentries. To this extent, the completed ratings matrix R1 is a densematrix. Accordingly, using the completed ratings matrix R1 to identifythe user attribute affinities involves a greater number ofcomputations—and is thus more computationally expensive—than multiplyingthe sparse ratings matrix R by the item-attribute matrix 208. Consider,for instance, that a computational cost for the affinity module 122 tomultiply the completed ratings matrix R1 with the item-attribute matrix208 is proportional to the following:C _(R1 Mult)=(M×k×N)+(M×N×A)

Here, the term C_(R1 Mult) represents the computational cost of usingthe completed ratings matrix R1 to compute the attribute affinity scores212. Further, the product of M×k×N represents a computing cost of theaffinity module 122 to multiply the first and second elemental matricesU and V, respectively. In contrast, the product of M×N×A represents acomputing cost of the affinity module 122 to multiply the completedratings matrix R1 with the item-attribute matrix 208.

Broadly speaking, matrix multiplication is associative. The affinitymodule 122 is configured to take advantage of this to compute theattribute affinity scores 212, rather than multiply the matrices of thisexample as discussed just above. Instead of multiplying the first andsecond elemental matrices U and V, and then multiplying the result bythe item-attribute matrix 208, I, to produce a product of (U×V)×I, forinstance, the affinity module first multiplies the second elementalmatrix V by the item-attribute matrix 208, I. The affinity module 122then multiplies this product by the first elemental matrix U, such thatU×(V×I). The affinity module 122 then determines the attribute affinityscores 212 by computing an LLR with the product of U×(V×I). By computingthe attribute affinity scores 212 in this way, a computing cost to theaffinity module 122 may be proportional to the following expression:C _(R1 Mult)=(k×N×A)+(M×k×A)=k×A×(M+N)

Here, the term C_(R1 Mult) represents the computational cost of usingthe completed ratings matrix R1 to compute the attribute affinity scores212, though computed according to the just described order to takeadvantage of matrix multiplication being associative. The product ofk×N×A represents a computing cost of the affinity module 122 to multiplythe second elemental matrix V by the item-attribute matrix 208.Additionally, the product of k×N×A represents a computing cost of theaffinity module 122 to determine the final result, e.g., the attributeaffinity scores 212.

Having described example procedures in accordance with one or moreimplementations, consider now an example system and device that can beutilized to implement the various techniques described herein.

Example System and Device

FIG. 7 illustrates an example system generally at 700 that includes anexample computing device 702 that is representative of one or morecomputing systems and/or devices that may implement the varioustechniques described herein. This is illustrated through inclusion ofthe recommendation manager module 114 and the communication module 112.The computing device 702 may be, for example, a server of a serviceprovider, a device associated with a client (e.g., a client device), anon-chip system, and/or any other suitable computing device or computingsystem.

The example computing device 702 as illustrated includes a processingsystem 704, one or more computer-readable media 706, and one or more I/Ointerfaces 708 that are communicatively coupled, one to another.Although not shown, the computing device 702 may further include asystem bus or other data and command transfer system that couples thevarious components, one to another. A system bus can include any one orcombination of different bus structures, such as a memory bus or memorycontroller, a peripheral bus, a universal serial bus, and/or a processoror local bus that utilizes any of a variety of bus architectures. Avariety of other examples are also contemplated, such as control anddata lines.

The processing system 704 is representative of functionality to performone or more operations using hardware. Accordingly, the processingsystem 704 is illustrated as including hardware elements 710 that may beconfigured as processors, functional blocks, and so forth. This mayinclude implementation in hardware as an application specific integratedcircuit or other logic device formed using one or more semiconductors.The hardware elements 710 are not limited by the materials from whichthey are formed or the processing mechanisms employed therein. Forexample, processors may be comprised of semiconductor(s) and/ortransistors (e.g., electronic integrated circuits (ICs)). In such acontext, processor-executable instructions may beelectronically-executable instructions.

The computer-readable storage media 706 is illustrated as includingmemory/storage 712. The memory/storage 712 represents memory/storagecapacity associated with one or more computer-readable media. Thememory/storage component 712 may include volatile media (such as randomaccess memory (RAM)) and/or nonvolatile media (such as read only memory(ROM), Flash memory, optical disks, magnetic disks, and so forth). Thememory/storage component 712 may include fixed media (e.g., RAM, ROM, afixed hard drive, and so on) as well as removable media (e.g., Flashmemory, a removable hard drive, an optical disc, and so forth). Thecomputer-readable media 706 may be configured in a variety of other waysas further described below.

Input/output interface(s) 708 are representative of functionality toallow a user to enter commands and information to computing device 702,and also allow information to be presented to the user and/or othercomponents or devices using various input/output devices. Examples ofinput devices include a keyboard, a cursor control device (e.g., amouse), a microphone, a scanner, touch functionality (e.g., capacitiveor other sensors that are configured to detect physical touch), a camera(e.g., which may employ visible or non-visible wavelengths such asinfrared frequencies to recognize movement as gestures that do notinvolve touch), and so forth. Examples of output devices include adisplay device (e.g., a monitor or projector), speakers, a printer, anetwork card, tactile-response device, and so forth. Thus, the computingdevice 702 may be configured in a variety of ways as further describedbelow to support user interaction.

Various techniques may be described herein in the general context ofsoftware, hardware elements, or program modules. Generally, such modulesinclude routines, programs, objects, elements, components, datastructures, and so forth that perform particular tasks or implementparticular abstract data types. The terms “module,” “functionality,” and“component” as used herein generally represent software, firmware,hardware, or a combination thereof. The features of the techniquesdescribed herein are platform-independent, meaning that the techniquesmay be implemented on a variety of commercial computing platforms havinga variety of processors.

An implementation of the described modules and techniques may be storedon or transmitted across some form of computer-readable media. Thecomputer-readable media may include a variety of media that may beaccessed by the computing device 702. By way of example, and notlimitation, computer-readable media may include “computer-readablestorage media” and “computer-readable signal media.”

“Computer-readable storage media” may refer to media and/or devices thatenable persistent and/or non-transitory storage of information incontrast to mere signal transmission, carrier waves, or signals per se.Thus, computer-readable storage media refers to non-signal bearingmedia. The computer-readable storage media includes hardware such asvolatile and non-volatile, removable and non-removable media and/orstorage devices implemented in a method or technology suitable forstorage of information such as computer readable instructions, datastructures, program modules, logic elements/circuits, or other data.Examples of computer-readable storage media may include, but are notlimited to, RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, digital versatile disks (DVD) or other optical storage, harddisks, magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or other storage device, tangible media, orarticle of manufacture suitable to store the desired information andwhich may be accessed by a computer.

“Computer-readable signal media” may refer to a signal-bearing mediumthat is configured to transmit instructions to the hardware of thecomputing device 702, such as via a network. Signal media typically mayembody computer readable instructions, data structures, program modules,or other data in a modulated data signal, such as carrier waves, datasignals, or other transport mechanism. Signal media also include anyinformation delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media include wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared, and other wireless media.

As previously described, hardware elements 710 and computer-readablemedia 706 are representative of modules, programmable device logicand/or fixed device logic implemented in a hardware form that may beemployed in some embodiments to implement at least some aspects of thetechniques described herein, such as to perform one or moreinstructions. Hardware may include components of an integrated circuitor on-chip system, an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), a complex programmable logicdevice (CPLD), and other implementations in silicon or other hardware.In this context, hardware may operate as a processing device thatperforms program tasks defined by instructions and/or logic embodied bythe hardware as well as a hardware utilized to store instructions forexecution, e.g., the computer-readable storage media describedpreviously.

Combinations of the foregoing may also be employed to implement varioustechniques described herein. Accordingly, software, hardware, orexecutable modules may be implemented as one or more instructions and/orlogic embodied on some form of computer-readable storage media and/or byone or more hardware elements 710. The computing device 702 may beconfigured to implement particular instructions and/or functionscorresponding to the software and/or hardware modules. Accordingly,implementation of a module that is executable by the computing device702 as software may be achieved at least partially in hardware, e.g.,through use of computer-readable storage media and/or hardware elements710 of the processing system 704. The instructions and/or functions maybe executable/operable by one or more articles of manufacture (forexample, one or more computing devices 702 and/or processing systems704) to implement techniques, modules, and examples described herein.

The techniques described herein may be supported by variousconfigurations of the computing device 702 and are not limited to thespecific examples of the techniques described herein. This functionalitymay also be implemented all or in part through use of a distributedsystem, such as over a “cloud” 714 via a platform 716 as describedbelow.

The cloud 714 includes and/or is representative of a platform 716 forresources 718. The platform 716 abstracts underlying functionality ofhardware (e.g., servers) and software resources of the cloud 714. Theresources 718 may include applications and/or data that can be utilizedwhile computer processing is executed on servers that are remote fromthe computing device 702. Resources 718 can also include servicesprovided over the Internet and/or through a subscriber network, such asa cellular or Wi-Fi network.

The platform 716 may abstract resources and functions to connect thecomputing device 702 with other computing devices. The platform 716 mayalso serve to abstract scaling of resources to provide a correspondinglevel of scale to encountered demand for the resources 718 that areimplemented via the platform 716. Accordingly, in an interconnecteddevice embodiment, implementation of functionality described herein maybe distributed throughout the system 700. For example, the functionalitymay be implemented in part on the computing device 702 as well as viathe platform 716 that abstracts the functionality of the cloud 714.

CONCLUSION

Although the invention has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the invention defined in the appended claims is not necessarilylimited to the specific features or acts described. Rather, the specificfeatures and acts are disclosed as example forms of implementing theclaimed invention.

What is claimed is:
 1. In a digital medium environment to providedigital content recommendations to client device users, a methodimplemented by a computing device, the method comprising: determining,by the computing device, user affinity scores for a plurality ofattributes of digital content, the user affinity scores determined basedon a completed user-item matrix and an item-attribute matrix, thecompleted user-item matrix augmenting a sparse user-item matrix havingcells that indicate strength-of-interaction measures of a client deviceuser of the client device users with digital content items of a catalogof digital content items, the strength-of-interaction measures obtainedfrom interaction data of the client device user that describesinteractions of the client device user with the digital content items ofthe catalog of digital content items, the completed user-item matrixhaving additional cells that indicate predicted strength-of-interactionmeasures for the client device user obtained from additional interactiondata describing interactions of an additional client device user withadditional digital content items, the additional client device userdetermined similar to the client device user and the client device userhas not consumed the additional digital content items; identifying, bythe computing device and from the catalog of digital content items, afirst set of digital content items associated with a first attribute ofthe plurality of attributes using the completed user-item matrix insteadof the sparse user-item matrix, the first set of digital content itemsidentified based on detecting an association between the first attributeand metadata included in the first set of digital content items;ranking, by the computing device, the first set of digital content itemsbased on the user affinity scores and additional associations detectedbetween the plurality of attributes and the metadata; selecting, by thecomputing device, a subset of the first set of digital content itemsbased on the ranking, the subset of digital content items including atleast one digital content item that has not been consumed by the clientdevice users; generating, by the computing device, at least onerecommendation for the subset of digital content items; and providing,by the computing device via a network, the at least one recommendationto a client device.
 2. A method as described in claim 1, furthercomprising: generating a justification associated with the at least onerecommendation based on the first attribute; and providing thejustification with the at least one recommendation.
 3. A method asdescribed in claim 1, wherein the at least one recommendation isprovided to the client device substantially in real time after a requestfor recommendations is received.
 4. A method as described in claim 1,wherein the first attribute is a highest scoring of the plurality ofattributes according to the user affinity scores.
 5. A method asdescribed in claim 1, further comprising performing the identifying,ranking, selecting, and generating for at least one other attribute ofthe plurality of attributes to recommend a second set of digital contentitems associated with the at least one other attribute.
 6. A method asdescribed in claim 5, further comprising receiving a request from theclient device for recommendations, the request indicating a number ofattributes for which to perform the identifying, ranking, selecting, andgenerating.
 7. A method as described in claim 1, wherein a number ofdigital content items selected for the subset is based on a request fromthe client device for the at least one recommendation, the requestindicating the number of digital content items to select for the subset.8. A method as described in claim 1, wherein determining the useraffinity scores includes: generating the completed user-item matrix; andgenerating the item-attribute matrix having cells that indicateassociations between the plurality of attributes and the digital contentitems of the catalog.
 9. A method as described in claim 1, whereindetermining the user affinity scores further includes: multiplying thecompleted user-item matrix and the item-attribute matrix; and applying aLog-Likelihood Ratio to the multiplied matrices to compute the useraffinity scores.
 10. A method as described in claim 1, furthercomprising: determining a latent factor vector for the client deviceuser of the client device users based on the user affinity scores; andextracting latent factor vectors for the first set of digital contentitems from the completed user-item matrix using a matrix factorizationalgorithm.
 11. A method as described in claim 10, further comprisingcomputing dot products of the latent factor vector of the client deviceuser and the latent factor vectors of the first set of digital contentitems.
 12. A method as described in claim 11, wherein the ranking isbased on the dot products.
 13. A system comprising: an affinity moduleimplemented at least partially in hardware of at least one computingdevice to determine user affinity scores for a plurality of attributesof digital content, the user affinity scores determined based on acompleted user-item matrix and an item-attribute matrix, the completeduser-item matrix augmenting a sparse user-item matrix having cells thatindicate strength-of-interaction measures of a client device user ofclient device users with digital content items of a catalog, thestrength-of-interaction measures obtained from interaction data of theclient device user that describes interactions of the client device userwith the digital content items of the catalog, the completed user-itemmatrix having additional cells that indicate predictedstrength-of-interaction measures for the client device user obtainedfrom additional interaction data describing interaction of an additionalclient device user with additional digital content items, the additionalclient device user determined similar to the client device user and theclient device user has not consumed the additional digital contentitems; a collaborative filtering module implemented at least partiallyin the hardware of the at least one computing device to filter thedigital content items of the catalog for recommendation by: identifyinga first set of the digital content items that are associated with afirst attribute of the plurality of attributes using the completeduser-item matrix instead of the sparse user-item matrix, the firstattribute used for the identifying being based on the user affinityscores; ranking the first set of digital content items based on the useraffinity scores and associations of the first set of digital contentitems with the plurality of attributes; and selecting a subset ofdigital content items from the first set of digital content items basedon the ranking to form at least one recommendation for the subset ofdigital content items, the subset of digital content items including atleast one digital content item that has not been consumed by the clientdevice users.
 14. A system as described in claim 13, wherein thecollaborative filtering module is further configured to filter thedigital content items by identifying at least a second set of thedigital content items that are associated with at least one otherattribute of the plurality of attributes, and performing the ranking andselecting in relation to the at least second set of digital contentitems.
 15. A system as described in claim 13, wherein the firstattribute corresponds to a highest scoring of the plurality ofattributes according to the user affinity scores.
 16. A system asdescribed in claim 13, wherein the collaborative filtering module isfurther configured to generate a justification associated with the atleast one recommendation based on the first attribute.
 17. A system asdescribed in claim 13, further comprising: a matrix generation moduleimplemented at least partially in the hardware of the at least onecomputing device to: generate the completed user-item matrix; andgenerate the item-attribute matrix having cells that indicate detectedassociations between the plurality of attributes and the digital contentitems of the catalog; and a user-attribute scoring module implemented atleast partially in the hardware of the at least one computing device todetermine the user affinity scores by: multiplying the completeduser-item matrix and the item-attribute matrix; and applying aLog-Likelihood Ratio to the multiplied matrices to compute the useraffinity scores.
 18. In a digital medium environment to provide digitalcontent recommendations to client device users, a method implemented bya computing device, the method comprising: generating, by the computingdevice, a completed user-item matrix, the completed user-item matrixaugmenting a sparse user-item matrix having cells that indicatestrength-of-interaction measures of a client device user with a catalogof digital content items, the completed user-item matrix havingadditional cells that indicate predicted strength-of-interactionmeasures for the client device user obtained from interaction datadescribing interactions of an additional client device user withadditional digital content items, the additional client device userdetermined similar to the client device user and the client device userhas not consumed the additional digital content items; generating, bythe computing device, an item-attribute matrix having cells thatindicate associations of digital content items of the catalog with aplurality of attributes of digital content; computing, by the computingdevice, user affinity scores based on the completed user-item matrixinstead of the sparse user-item matrix and the item-attribute matrix;identifying, by the computing device and from the catalog, a first setof digital content items associated with a first attribute of theplurality of attributes, the first attribute used for the identifyingbeing based on the user affinity scores; determining, by the computingdevice, a latent factor vector for the client device user based on theuser affinity scores; extracting, by the computing device, latent factorvectors for the first set of digital content items from the user-itemmatrix using matrix factorization; computing, by the computing device,dot products of the latent factor vector of the client device user andthe latent factor vectors of the first set of digital content items;selecting, by the computing device, a subset of digital content itemsfrom the first set of digital content items based on the dot products,the subset of digital content items including at least one digitalcontent item that has not been consumed by the client device users; andproviding, by the computing device and to the client device via anetwork, at least one recommendation for the subset of digital contentitems.
 19. A method as described in claim 18, further comprisingranking, by the computing device, the first set of digital content itemsbased at least in part on the user affinity scores.
 20. A method asdescribed in claim 18, further comprising: generating a justificationassociated with the at least one recommendation based on the firstattribute; and providing the justification with the at least onerecommendation.